{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Observation Simulation Tutorial\n",
    "\n",
    "This tutorial provides some step-by-step examples of how to simulate Fermi observations using the [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt) application.\n",
    "\n",
    "For example: a Guest Investigator planning a proposal may need to make a realistic assessment of the detection significance achievable within a given time interval or the ability to constrain source model parameters for a given source intensity. More subtle issues may also be explored: e.g. how sensitive a given observation may be to pulsed emission of a given flux and modulation depth, spatially extended emission, or pointed versus survey mode observation scenarios.\n",
    "\n",
    "The [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt) tool (and related [gtorbsim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtorbsim.txt) tool) produce Fermi simulated data files. Which can then be analyzed with the Fermitools following the same procedure as with the real data.\n",
    "\n",
    "This tutorial assumes the user has a spacecraft data file already generated with the [gtorbsim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtorbsim.txt) tool, or that he or she has obtained a sample spacecraft data file from the [FSSC website](https://fermi.gsfc.nasa.gov/ssc/) (step 4). Alternatively, for basic zenith-rocking scenarios, one can be created \"on the fly\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Steps**:\n",
    "\n",
    "1. **Download the latest model for the isotropic background.**\n",
    "\n",
    "    To simulate the background, you will need to download the necessary files that specify the Galactic interstellar emission and the isotropic component. For Pass 8 this is more complicated than with previous data releases because of the number of data selection options. In all cases the Galactic file is `gll_iem_v06.fits`. For standard analysis using SOURCE class (FRONT+BACK), the isotropic spectral template is `iso_P8R3_SOURCE_V2.txt`.\n",
    "\n",
    "\n",
    "2. **Prepare XML source library files**\n",
    "\n",
    "    These are XML files containing the source model definitions used by [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt).\n",
    "\n",
    "\n",
    "3. **Create a list of XML input file names**\n",
    "\n",
    "    Various XML source libraries already exist. In addition to creating custom libraries, you can make sources in these libraries available by including the full paths to the files in this list.\n",
    "\n",
    "\n",
    "4. **Create a list of sources**\n",
    "\n",
    "    These are the sources to be modeled.\n",
    "\n",
    "\n",
    "5. **Specify or create a pointing and livetime history file**\n",
    "\n",
    "    You can: use an existing pointing and livetime history file; create one using the [gtorbsim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtorbsim.txt) tool; or define a pointing strategy and let [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt) compute one.\n",
    "    \n",
    "    \n",
    "6. **Run [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Download the latest model for the isotropic background\n",
    "\n",
    "When you use the latest Galactic emission model `gll_iem_v06.fits` in a likelihood analysis, you will want to use a template for the isotropic diffuse emission, which includes the residual cosmic-ray background. For standard analysis you will need to download `iso_P8R3_SOURCE_V2.txt`. This is valid only for the `P8R3_SOURCE_V2` response function and only for data sets with front + back events combined.\n",
    "\n",
    "For isotropic spectral templates for other data selections, please see the Background Models page. The file `isotropic_allsky.fits` is also required. This is simply an image with all pixels set to a value of `1`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Prepare the XML source library files\n",
    "\n",
    "These files contain the XML definitions of the sources that are used by [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt). These files by themselves are not valid XML documents, but rather they contain XML \"source_library\" tags that are compiled in memory into a valid XML document that is then parsed by the software. It should be noted that these files are analogous to, but unfortunately not compatible with gtlike xml source model files.\n",
    "\n",
    "The following example shows how to define point sources with simple and broken power-law spectra, diffuse sources using FITS image files as a templates in two different ways, and an isotropic diffuse source.\n",
    "\n",
    "The following xml files can be extracted from this document and edited with a text editor, or created with the model editor tool. Model editor creates models for **gtobssim** that are also available for **gtlike**. To use model editor tool, type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ModelEditor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "in the command line. The tool has its own help."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we specified source parameters derived from the Third EGRET Catalog, and incorporating the current Fermi background models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('./data/example1.xml', 'w') as file:\n",
    "    file.write('''<source_library title=\"example_sources\">\n",
    "\n",
    "<source name=\"example_3C279\" flux=\"0.0348\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<particle name=\"gamma\">\n",
    "<power_law emin=\"20.0\" emax=\"200000.\" gamma=\"1.96\"/>\n",
    "</particle>\n",
    "<celestial_dir ra=\"193.98\" dec=\"-5.82\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"example_3C273\" flux=\"0.0196\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<particle name=\"gamma\">\n",
    "<power_law emin=\"20.0\" emax=\"200000.\" gamma=\"2.58\"/>\n",
    "</particle>\n",
    "<celestial_dir ra=\"187.25\" dec=\"2.17\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"crab_pulsar\" flux=\"0.154\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<particle name=\"gamma\">\n",
    "<power_law emin=\"20\" emax=\"1000000.\" gamma=\"2.19\" ebreak=\"1500\" gamma2=\"4.89\"/>\n",
    "</particle>\n",
    "<celestial_dir ra=\"83.57\" dec=\"22.01\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"IsotropicDiffuse\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<SpectrumClass name=\"FileSpectrumMap\" params=\"flux=0.,fitsFile=isotropic_allsky.fits,specFile=iso_P8R3_SOURCE_V2.txt\"/>\n",
    "<use_spectrum frame=\"galaxy\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"GalacticDiffuse\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<SpectrumClass name=\"MapCube\" params=\"flux=6.508, fitsFile=gll_iem_v06.fits\"/>\n",
    "<use_spectrum frame=\"galaxy\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"standard_diffuse_components\">\n",
    "<nestedSource sourceRef=\"GalacticDiffuse\" />\n",
    "<nestedSource sourceRef=\"IsotropicDiffuse\" />\n",
    "</source>\n",
    "\n",
    "</source_library>\n",
    "''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```xml\n",
    "<source_library title=\"example_sources\">\n",
    "\n",
    "<source name=\"example_3C279\" flux=\"0.0348\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<particle name=\"gamma\">\n",
    "<power_law emin=\"20.0\" emax=\"200000.\" gamma=\"1.96\"/>\n",
    "</particle>\n",
    "<celestial_dir ra=\"193.98\" dec=\"-5.82\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"example_3C273\" flux=\"0.0196\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<particle name=\"gamma\">\n",
    "<power_law emin=\"20.0\" emax=\"200000.\" gamma=\"2.58\"/>\n",
    "</particle>\n",
    "<celestial_dir ra=\"187.25\" dec=\"2.17\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"crab_pulsar\" flux=\"0.154\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<particle name=\"gamma\">\n",
    "<power_law emin=\"20\" emax=\"1000000.\" gamma=\"2.19\" ebreak=\"1500\" gamma2=\"4.89\"/>\n",
    "</particle>\n",
    "<celestial_dir ra=\"83.57\" dec=\"22.01\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"IsotropicDiffuse\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<SpectrumClass name=\"FileSpectrumMap\" params=\"flux=0.,fitsFile=isotropic_allsky.fits,specFile=iso_P8R3_SOURCE_V2.txt\"/>\n",
    "<use_spectrum frame=\"galaxy\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"GalacticDiffuse\">\n",
    "<spectrum escale=\"MeV\">\n",
    "<SpectrumClass name=\"MapCube\" params=\"flux=6.508, fitsFile=gll_iem_v06.fits\"/>\n",
    "<use_spectrum frame=\"galaxy\"/>\n",
    "</spectrum>\n",
    "</source>\n",
    "\n",
    "<source name=\"standard_diffuse_components\">\n",
    "<nestedSource sourceRef=\"GalacticDiffuse\" />\n",
    "<nestedSource sourceRef=\"IsotropicDiffuse\" />\n",
    "</source>\n",
    "\n",
    "</source_library>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the FileSpectrumMap object, setting the `flux` parameter to `0` will integrate the specFile assuming it has two columns of energy in MeV and dN/dE/dt/dA in units of photons/m2/s/MeV, and use that integral as the flux.\n",
    "\n",
    "For the MapCube, it is necessary to explicitly give the flux. In this example, the flux is set to the integral of the Galactic Diffuse model map cube.\n",
    "\n",
    "The name of the xml file should have the `xml` extension. Other source class examples can be found in the Sources available to **gtobssim** section."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Create a list of XML input file names\n",
    "\n",
    "When [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt) is run, a list of XML input file names may be provided. This list is an ascii file giving full or relative paths to the XML files containing the source definitions. If this list is omitted (by entering `none`), then the following XML files are used by default:\n",
    "```\n",
    "$(FERMI_DIR)/xml/fermi/observationSim/3EG_catalog_20-1e6MeV.xml\n",
    "$(FERMI_DIR)/xml/fermi/observationSim/obsSim_source_library.xml\n",
    "$(FERMI_DIR)/xml/fermi/GRBobs/GRB_user_library.xml\n",
    "```\n",
    "If there is only one XML file, then the name of that file may be given rather than the ascii list."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Create a list of sources\n",
    "\n",
    "This is an ascii file containing a list of the sources that are to be modeled. For example, assuming the above `source_library` is used, then to model 3C 279, 3C 273, and the interstellar and extragalactic diffuse components, this file would contain\n",
    "```\n",
    "standard_diffuse_components\n",
    "example_3C279\n",
    "example_3C273\n",
    "#crab_pulsar\n",
    "```\n",
    "Lines beginning with \"#\" are treated as comments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Provide a pointing and livetime history file\n",
    "\n",
    "One has the option of providing a pointing and livetime history file to gtobssim or letting gtorbsim calculate the spacecraft orbit and attitude. The pointing and livetime history file may be a FITS file with this format, or it may be an ascii file with a format defined by this code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Run gtobssim\n",
    "\n",
    "Assuming the above XML file is `example_library.xml`, the list of sources is called `source_names.dat`, and they are both in the current directory folder `data`, one can run [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt) like this (for a one day long simulation):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gtobssim\n",
    "    ./data/example_library.xml\n",
    "    ./data/source_names.txt\n",
    "    none\n",
    "    test\n",
    "    86400\n",
    "    INDEF\n",
    "    no\n",
    "    P8R3_SOURCE_V2\n",
    "    293049"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the startdate parameter has been set to 2009-01-20. Since the photon arrival times are referred to the mission start date of 2001-01-01 00:00:00, the earliest possible photon arrival time in the resulting events file would be 254102400 MET seconds.\n",
    "\n",
    "In the example, no spacecraft data file was entered in \"Pointing history file.\" In this case the spacecraft data file is generated directly by gtobssim and is named `test_scData_0000.fits`.\n",
    "\n",
    "The event file is named, in this case, `test_events_0000.fits`. The map that can be created using gtbin could be downloaded from here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Related Tools\n",
    "\n",
    "* [gtobssim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtobssim.txt)\n",
    "* [gtorbsim](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtorbsim.txt)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
